1. Field of the Invention
The present invention relates to an optical apparatus having a line of sight detection device for performing a predetermined operation based on a detected line of sight of a user.
2. Related Background Art
Conventionally, there are provided various, so called, line of sight (visual axis) detection devices (for example, an eye camera) for detecting which position a photographer is observing on an observed surface. For example, in U.S. Pat. No. 5,486,892 issued on Jan. 23, 1996, a gazing point is calculated by using a cornea reflection image generated by reflected light from the cornea and an image-formed position on a pupil with projecting parallel flux from a light source on a front zone of an eyeball of the photographer. In addition, the publication discloses an example of adjusting the focus of a photo taking lens automatically by using gazing point information of a photographer by arranging a gazing point detection device in a single lens reflex camera.
FIG. 47 is a schematic diagram of a line of sight detection optical system arranged in a single lens reflex camera. This drawing shows a photo taking lens 1, a main mirror 2, a focusing plate 7, and a pentaprism 8. A photographer observes an object within a viewfinder by bringing his or her eye (an eyeball 15 of the photographer) close to an eyepiece lens 11. Each of light sources 13a and 13b, such as a light emitting diode emits an infrared light to which the photographer is not sensitive. A part of an illumination light reflected by the eyeball of the photographer is condensed on an image sensor 14 through a receptor lens 12.
FIG. 48A is a schematic diagram of an eyeball image which is projected on the image sensor 14. This drawing shows cornea reflection images 52a and 52b of the infrared light emitting diodes 13a and 13b, white 50 of an eyeball, pupil 51, and skin 53 surrounding the eye. FIG. 48B shows the intensity of an output signal from a line (section (E)-(E')) on the image sensor 14. The eyeball image is characterized by the highest luminance in the relatively smaller areas of the cornea reflection images 52a and 52b of the infrared light emitting diodes, and by the lowest luminance level in the relatively wide area of the pupil 51 due to its reflectance which is extremely low. A luminance level of the white 50 is between the levels of cornea reflection image and the pupil, and that of the skin 53 depends on outdoor daylight or illumination conditions.
FIG. 49 is a diagram describing the principle of a line of sight detection. This drawing shows an eyeball 15 of a photographer, a cornea 16, and an iris 17.
By using these drawings, a line of sight detection method is described below.
An infrared light emitted from the light source 13b irradiates the cornea 16 of the eyeball 15. A cornea reflection image d (a virtual image) formed by a part of the infrared light reflected on a surface of the cornea 16 is condensed by the receptor lens 12 to form an image at position d' on the image sensor 14. In the same manner, infrared light emitted from the light source 13a irradiates the cornea 16 of the eyeball 15. A cornea reflection image e formed by a part of the infrared light reflected on a surface of the cornea 16 is condensed by the receptor lens 12 to form an image at position e' on the image sensor 14.
Flux from edge portions a and b of the iris 17 forms an image of the edge portions a and b at positions a' and b' on the image sensor 14 via the receptor lens 12. If a rotation angle .theta. of an optical axis of the eyeball 15 to the optical axis on the receptor lens 12 is smaller, a coordinate xc of a center position c of the pupil 19 is expressed by the following: EQU xc.apprxeq.(xa+xb)/2
where the x coordinates of the edge portions a and b of the iris 17 are xa and xb, respectively.
An x coordinate of a middle point between the cornea reflection images d and e almost matches s coordinate xo of a center of curvature o of the cornea 16. Accordingly, a rotation angle .theta.x of an optical axis 15a of the eyeball 15 almost satisfies the following relation: EQU OC.times.SIN.theta.x.apprxeq.(xd+xe)/2-xc (1)
where xd and xe denote the x coordinates of positions d and e at which the cornea reflection images are generated and OC denotes a standard distance between the center of curvature o of the cornea and the center c of the pupil 19. Therefore, as shown in FIG. 48A, the rotation angle .theta. of the optical axis 15a of the eyeball 15 can be obtained by detecting the positions of the characteristic points (the cornea reflection image and the center of the pupil) of the eyeball 15 projected on the image sensor 14.
The rotation angle of the optical axis 15a of the eyeball is calculated as follows based on the expression (1): EQU .beta..times.OC.times.SIN.theta.x.apprxeq.{(xpo-.delta.x)-xic}.times.pitch( 2) EQU .beta..times.OC.times.SIN.theta.y.apprxeq.{(ypo-.delta.y)-yic}.times.pitch( 3)
where .theta.x denotes a rotation angle of an optical axis of the eyeball in a plane (z-x) and .theta.y denotes a rotation angle of an optical axis of the eyeball in a plane (y-z). (xpo, ypo) are the coordinates of a middle point of two cornea reflection images on the image sensor 14 and (xic, yic) are the coordinates of a center of the pupil on the image sensor 14. The characters' "pitch" denotes a picture element pitch of the image sensor 14. Character .beta. indicates an image formation scale factor which depends on a position of the eyeball 15 with respect to the receptor lens 12; it is calculated as a function of an interval between two cornea reflection images, practically.
Characters .delta.x and .delta.y denote correction terms for correcting the coordinates of the middle point of the cornea reflection images, including a correction term for correcting an error generated by projection on the eyeball of the photographer not with parallel flux but with divergent flux, and as to .delta.y, a correction term for correcting offset components generated by projecting the eyeball of the photographer with divergent flux from a direction of a lower eyelid.
After the rotation angle (.theta.x, .theta.y) of the optical axis of the eyeball of the photographer is calculated, a gazing point (x, y) on an observed plane (focusing plate) of the photographer is expressed as follows, if a camera is set horizontally: EQU x=m* (.theta.x+.DELTA.) (4a) EQU y=m* .theta.y (5a)
where the x direction corresponds to a horizontal direction to the photographer if the camera is set horizontally and the y direction corresponds to a vertical direction to the photographer if the camera is set horizontally. Character m indicates a transformation coefficient for transforming the rotation angle of the eyeball to coordinates on the focusing plate, and character .DELTA. indicates an angle formed by the optical axis 15a of the eyeball and the visual axis (gazing point). Generally, it is already known that the visual axis of an actual view of an observer is displaced relative to the rotation angle of the eyeball by approximately 5.degree. horizontally to the observer, but it is not displaced almost at all relative to the rotation angle in a vertical direction.
An explanation will be provided about an operation of a focus detection means conventionally arranged in a camera. The focus detection means has a plurality of focus detection operation modes; generally, a focus detecting operation mode for a static object and a focus detecting operation mode for a dynamic object.
The focus detecting operation mode for a static object is characterized by omitting any focus detecting operation once an in-focus state is detected.
The focus detecting operation mode for a dynamic object is characterized by continuing the focus detecting operation also after an in-focus state is detected.
Now, the line of sight and focus detecting operations of a camera having the line of sight detection means are described below by the focus detecting operation modes.
1) Focus detecting operation mode for a static object PA0 2) Focus detecting operation mode for a dynamic object
First, if a release button is kept pressed halfway (pressed by a first stroke), the line of sight detection means operates to calculate a gazing point within a visual field of a viewfinder of a photographer before detecting the focus. This point is represented by coordinates in the visual field of the viewfinder.
Based on the coordinates in the visual field of the viewfinder, the location of a focus detecting area corresponding to it is determined. For this obtained focus detecting area, a focus state is detected by a focus detection means and a photo taking lens is driven to an in-focus state based on the information.
After the focus detecting area is determined by the line of sight detection means as described above, a focusing operation is performed only once on the basis of the focus state of the focus detecting area.
If a release button is kept pressed halfway (pressed by a first stroke), the line of sight detection means operates to determine a focus detecting area. Afterwards, a lens driving operation is continued so as to keep an in-focus state based on only focus information of the focus detecting area.
As another aspect of the focus detecting operation mode for the above dynamic object, the focus detecting operation can be performed after determining a focus detecting area by detecting a line of sight every time before the focus detecting operation so that the focus detecting operation is linked with movement of the line of sight of a photographer due to movement of an object.
The position of line of sight of the photographer or conditions, however, are different at framing, for example, between a case of photographing a static object and a case of photographing a dynamic object. In other words, if a static object is photographed, a photographer may shift his or her line of sight to every detail of a scene to be photographed since he or she has a relatively long time. Therefore, the position of the line of sight sometimes exceeds a focus detecting area. In this case, if a focus detecting area is determined by ignoring the result of line of sight detection because the position of the line of sight deviates from the focus detecting area, the focus cannot be adjusted to the object intended by the photographer.
If a dynamic (moving) object is photographed, the line of sight of a photographer cannot follow the object and temporarily the position of the line of sight may deviate from the focus detecting area. It is, for example, a case that the photographer glances at another thing other than the object such as a background. In this case, if a focus detecting area is determined based on line of sight information, the result is contrary to the intention of the photographer because the lens is focused to the background.
In addition, also when a dynamic object is photographed, the line of sight of the photographer is different between a first line of sight detection and a second or later line of sight detection in repeating a line of sight detection and focus detection. In the first line of sight detection, the photographer shifts his or her line of sight to various places on a scene to be photographed since he or she has a relatively long time. In the second or later line of sight detection, he or she does not have enough time to see things other than the object. Accordingly, if the position of line of sight calculated by the line of sight detecting operation deviates significantly from the focus detecting area, it can be considered that the line of sight has shifted to the background by mistake.
In the same manner, the line of sight of the photographer changes in a different way between a case of one shot photographing and a case of continuous shot photographing.
Furthermore, it is also different between a case of a long distance to an object and a case of a short distance to an object. If the distance to the object is long, the gazing point is concentrated on a small range since an object image on the scene is relatively small. If it is short, however, the position of the line of sight shifts in a wide range since the object image on the scene is relatively large. Like this, if the conditions on photographing are different, the position of the line of sight of the photographer changes in a different manner during the taking of photos. Accordingly, if a focus detection area is selected always under the same conditions, a correct focus area cannot be determined.
In U.S. application Ser. No. 08/348,142 filed on Nov. 23, 1994, now U.S. Pat. No. 5,708,862, it is proposed that, based on information about a line of sight direction of an eyeball which is detected continuously, a gazing point within an observation range of a subject is calculated from a line of sight direction which is pointed for more than or equal to a predetermined time within a certain time or a line of sight direction pointed with more than or equal to a predetermined frequency.
In U.S. application Ser. No. 08/473,991 filed on Jun. 7, 1995, and refiled as Ser. No. 08/819,134 on Mar. 17, 1997, there is provided a single lens reflex camera having a line of sight input function of checking a depth-of-field by an aperture adjusting operation for a lens mounted on a camera when a photographer gazes at a line of sight input index arranged within a visual field of the viewfinder.
In the above conventional embodiments, however, the intention of a user sometimes cannot be reflected properly without an appropriate determination for some target objects observed in a viewfinder or scenes since determination standards on a gazing point are always identical in any conditions, though an automatic focus detection device of the camera is controlled with considering a part on which the line of sight resides for more than or equal to a predetermined time or with more than or equal to a predetermined frequency as a gazing point.